In recent years with advancements in digital imaging, image sensors have become more popular for measuring macroscopic motions in a scene in three dimensions. However, estimating small motions in three dimensions using image sensors remains a difficult problem. Measuring micro-motions at macroscopic stand-off distances is not possible with conventional cameras and vision systems without using sophisticated optics and/or special purpose light sources. Furthermore, measuring multi-object or non-rigid motion is fundamentally more challenging than tracking a single object due to the considerably higher number of degrees of freedom, especially if the objects are devoid of high-frequency texture.
One approach for attempting to measure motion is a combination of two dimensional (2D) optical flow and changes in scene depths (sometimes referred to as scene flow). In this approach, both optical flow and depths are calculated to attempt to measure scene motion. For example, depth can be calculated using stereo cameras or an RGB-D camera. As another example, light field cameras have been used for recovering depths for calculating scene flow
Light field data has also been used for attempting to recover a camera's motion (i.e., ego-motion of the camera), and to compute three dimensional (3D) scene reconstructions via structure-from-motion techniques. These techniques are based on a constraint relating camera motion and light fields, and recover six degree-of-freedom camera motion from light fields, which is an over-constrained problem. However, these techniques are not suited to detecting object motion in a scene (e.g., by determining 3D non-rigid scene motion at every pixel), which is under-constrained due to considerably higher number of degrees of freedom.
Accordingly, systems, methods, and media for determining object motion in three dimensions from light field image data are desirable.